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DETAILED ACTION 

Claim Rejections - 35 USC §112 

1 . The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

2. Claims 6,15 and 22 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. 

With regards to claims 6 and 15, the applicant makes reference to a number of 
intermediary images which are used to create the final image l F . These claims are 
dependent upon claims 1 and 10 respectively, which disclose a process including the 
step of generating a human interactive proof which employs the use of an image. It is 
unclear from the claim language as to which specific image is being employed, the final 
image or any of the intermediary images. 

Regarding claim 22, the claim discusses a system where the output is test image 
l F . This claim is dependent upon claim 1 0; however there is no mention in claim 1 0 of an 
image bearing this name. It is unclear from the claim language as to how the applicant 
arrived at this specific image from the dependency on claim 10. 

Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
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the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1-4, 7-11, 13, 17-19, 21 and 22 are rejected under 35 U.S.C. 103(a) as 
being unpatentable overTyree US PGPUB 2002/0120853 (hereinafter referred to as 
Tyree) in view of Greg Mori et al. "Estimating Human Body Configurations using Shape 
Context Matching" from the 2001 IEEE Computing Society's Conference on Computer 
Vision and Pattern Recognition (hereinafter referred to as Mori). 



Claim 1 of the application recites: 

A computer-implemented process for determining whether a computer user is a 
human or a computer program, comprising the process actions of: 

generating a human interactive proof ( [0034], [0039], [0072], [0103]) employing 
an image ([0029], [0102], [0103]) of one or more body parts wherein certain 
features thereof are at known locations in said image; 

requiring a computer user to locate ([0029]) at least one feature of said one or 
more body parts in the image; 

comparing the computer user's locations of said at least one feature of said one 
or more body parts to their actual location in the image; and 

determining whether the computer user is a human or a computer program 
([0042], [0072], [0074]). 



Tyree discloses the exemplary embodiment of an intelligence test which offers an 
image and prompts the user to answer a question about said image ([0102]), which is 
then compared with the stored answer ([0103]). It does not however discuss an image 
which contains body parts, and location of such parts. The security of any human 
interactive proof is contingent upon the ability of the algorithm to correctly identify 
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elements within the produced tests for comparison with user input. The Mori paper 
discloses an algorithm for locating keypoints within an undistorted image of a body 
(paragraph 1, pg 1). The basic image then would be understandable to the program 
prior to its distortion. After transformations (which are known to the program) are 
applied, the resultant image would not be identifiable by body/facial recognition software 
of the time. It would have been obvious to one of ordinary skill in the art at the time of 
invention to combine the two in order to increase the level of security offered by the 
"test", thus improving the marketability of the software product as a whole. 

Claim 2 of the application recites: 

The computer-implemented process of Claim 1 wherein said one or more 
body parts is a human face. 

The additional limitation of a human face is discussed by Mori (section 5, pg 5). 
Claim 3 of the application recites: 

The computer-implemented process of Claim 1 wherein said one or more 
body parts is an entire human body. 

The additional limitation of a human body is taught by Mori (section 1, paragraph 

i.pgi). 

Claim 4 of the application recites: 



The computer-implemented process of Claim 1 wherein said one or more 
body parts is an animal. 
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The additional limitation of an animal is disclosed by Mori (section 1, paragraph - !, 

pgi)- 

Claim 7 of the application recites: 

The computer-implemented process of Claim 1 wherein the process 
action for determining whether the computer user is a human or a computer 
program comprises using a comparison of the computer user's locations of said 
at least one feature of said one or more body parts to the location of said 
features in the image. 

Tyree teaches a system which includes a comparator for comparing the expected 
answer of a given "test" with the answer provided by the user (claim 18). Utilizing the 
images of body parts disclosed earlier by Mori, the challenge put forth by the validating 
program ([0102]) could reasonably consist of identifying and locating parts of the image. 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
to combine the system of Tyree with the recognition system of Mori, to improve the 
marketability of the software product by providing a slightly different form of test 
challenge. 

Claim 8 of the application recites: 

The computer-implemented process of Claim 1 wherein the computer- 
user-identified feature locations are specified by the user, using a computer 
pointing device. 
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The limitation of identifying locations in an image using an input device is met by 
Tyree who discloses querying a user for user input regarding the challenge, within an 
input field, which can reasonably be construed as the clickable region of an image map. 
Furthermore Tyree describes computer having input devices which include a mouse or 
other pointing device ([0096]), thus meeting the additional limitation of a computer 
pointing device. Mori's algorithm discloses automated feature location (section 1 , 
paragraph 1 , pg 1 ). It would have been obvious to one of ordinary skill in the art to 
combine the two to improve efficiency of the human interactive proof by allowing a user 
to click on features rather than typing them out. 

Claim 9 of the application recites; 

The computer-implemented process of Claim 8 wherein the computer 
Pointing device is one of: 
a mouse; and 
a digital pen. 

The limitation of a pointing device is taught by Tyree's disclosure of computer 
input devices ([0096]). 



Claim 10 of the application recites: 

A system for creating a Human Interactive Proof using an image of a face, 
the system comprising: 

a general purpose computing device ([0092] - [0095]); and 

a computer program comprising program modules executable by the computing 
device, wherein the computing device is directed by the program modules of the 
computer program ([0098] - [0099]) to, 
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generate a human interactive proof employing an image ([0029], [0034], [0072], 
[0103]) of a human face wherein certain features are at known locations in said 
image; 

require a computer user to locate ([0029]) certain features of said face in the 
image; 

compare the computer user's locations of said features of said face to their 
actual location in the image; and 

determine whether the computer user is a human or a bot ([0042], [0072], 
[0074]). 

Tyree discloses an intelligence test which offers an image to a potential user and 
prompts the user to answer a question about said image ([0102]), which is then 
compared with the stored answer ([0103]). It does not disclose an image which contains 
a human face, and location of facial features. Mori paper discloses an algorithm for 
locating keypoints within an undistorted image of a body (paragraph 1 , pg 1 ), which can 
be applied to facial recognition (item 1 , section 5, pg 5). The basic image then would be 
understandable to the program prior to its distortion. After transformations (which are 
known to the program) are applied, the resultant image would not be identifiable by 
body/facial recognition software of the time. It would have been obvious to one of 
ordinary skill in the art at the time of invention to combine the two in order to increase 
the level of security offered by the "test", thus improving the marketability of the 
software product as a whole. 

Claim 11 of the application recites: 

The system of Claim 10 wherein the image is automatically synthesized. 
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Tyree does not explicitly disclose automated image synthesis it does discuss a 
random test generator that determines an expected answer ([0039]). The Mori paper 
teaches an algorithm for automatically determining features on an image of a human 
body. It is necessary for the test generator of Tyree to have the capability of determining 
the answer to the test, in this case the location of body parts within the image. It would 
have been obvious to one of ordinary skill in the art at the time of the invention to 
combine the test generator of Tyree with the automated process of Mori in order to 
improve efficiency. 

Claim 13 of the application recites: 

The system of Claim 10 wherein the module to determine whether a 
computer user is a human or a bot determines that the computer user is a human 
if the computer user's locations of said features are within a given distance from 
their actual location. 

The only additional limitation claim 13 adds to claim 10 is that of the locations of 
features, which are input by the user, as being within a given distance from their actual 
location. With regards to a comparison of feature location, there are three possible 
outcomes for a user's estimated location: it is either within a given distance, outside a 
given distance, or at the origin/at the actual location. Since a comparison of the 
estimated features is being made to their actual location already it would be obvious to 
one of ordinary skill in the art at the time of the invention to conduct that comparison 
based on the estimate being within a given distance. 
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Claim 17 of the application recites: 

The system of Claim 1 0 wherein the determination of whether the user is 
a human or a computer program is made without human intervention. 

The limitation of lack of human intervention is taught by Tyree as a comparator 
component of the system ([0039]). 

Claim 18 of the application recites: 

The system of Claim 10 wherein the user points to the feature points with 
a computer input device. 

Additionally Claim 19 recites: 

The system of Claim 18 wherein the computer input device is a mouse. 

The Tyree publication discloses the limitation of identifying locations in an image 
using an input device by querying a user for input regarding the challenge, within an 
input field, which can reasonably be construed as the clickable region of an image map. 

Claim 21 of the application recites: 

The system of Claim 10 wherein the image size is 512 x 512 pixels. 

The examiner takes official notice that it would have been obvious to one of 
ordinary skill in the art of image manipulation at the time of the invention to generate, 
crop, resize, or otherwise manipulate the image of claim 1 0 to the size of 51 2 x 51 2 
pixels. 
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Claim 22 of the application recites: 

The system of Claim 10 wherein the output of the image generation 

module is test image IF with ground truth of face locations and facial feature 
locations. 

The scope of this claim is covered under claim 10's disclosure of generating an 
HIP employing the image of a human face and having known locations of features 
within the image. These feature locations are used by the system for comparison 
([0034]), to determine authentication and thus it is inherent that ground truth should be 
represented by these locations. This claim encompasses no additional matter and is 
therefore rejected under the same premise as claim 10. 



3. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Tyree in 
view of Mori and further in view of Luis von Ahn, et al "CAPTCHA: Using Hard Al 
Problems For Security" (hereinafter referred to as von Ahn), presented at the EuroCrypt 
2003 conference in Warsaw Poland. 

Claim 5 of the application recites: 

The computer-implemented process of Claim 1 wherein said 

determination of whether a computer user is a human or a computer program is 

used for one of : 

assigning an email account; 

validating an input in a poll; 

using a search engine; 

using a chat room; and 

accessing data on a website. ([0036]) 
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Tyree teaches the concept of using a human interactive proof to control access 
to network resources and storage ([0036]). The von Ahn paper goes further to describe 
the use of HIP for Online polls (pg 1 , section 1 , paragraph 2), Email account services 
(section 1 , paragraph 1 , pg 2), and Search Engines (section 1 , paragraph 2, pg 2). All of 
these are examples of network resources and storage facilities. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to utilize the HIP for 
controlling access to many types of internet accessible resources to expand the 
usefulness of the proof generating software and thus improve its overall marketability 

4. Claims 12, 16, 20 and 23-25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Tyree in view of Mortlock et al, US Patent 6,549,200 (hereinafter 
referred to as Mortlock). 

Claim 12 of the application recites: 

The system of Claim 10 wherein the image is a distorted face embedded 
in a cluttered background 

The applicant discloses that it is known in the art to put an image of a distorted 
word on a cluttered background image ([0006]). The applicant does not disclose the 
image of a face on a cluttered background. The Mortlock patent discloses the ability to 
create an image of a human head/face. It would have been obvious to one of ordinary 
skill in the art at the time of the invention to combine the applicant's disclosed image, 
with that of Mortlock's facial image in order to expand the variety of images that can be 
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used in an HIP system. Increasing the variety of usable images will increase security 
offered by the system and therefore its marketability. 

Claim 16 of the application recites: 

The system of Claim 10 wherein the image is generated to include at least 
one of: 

non-frontal faces; 

faces that are non-symmetrical; 

various lighting and shading conditions; and 

a background that contains face-like clutter. 

It has been established with regards to the previous claim that putting text on a 
distorted text ([006]), and that it would have been obvious to combine this with the facial 
image creation of Mortlock. Mortlock discloses the limitations of on-frontal faces (fig 
4Fig 3A, Fig 13), and pictures with varied lighting effects (column 13, lines 32-34). It 
would have additionally been obvious to one of ordinary skill in the art to one of ordinary 
skill in the art of graphic design, to manipulate the facial images of Mortlock to make 
them non-symmetrical. Including these features in the HIP would improve its ability to 
deter non-human users and thus improve it offers, and make it more marketable. 

Claim 20 of the application recites: 

The system of Claim 10 wherein the inputs to generate the image are a 

3D wire model of a generic head and a cylindrical texture map Tm of an arbitrary 

person. 
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Tyree discloses a computer means for creating human interactive proof images 
([0029], [0034], [0072], [0103]). It does not discuss the manner in which these images 
are created. The creation of 3-d images utilizing wire models was will known in the art of 
graphics creation at the time of invention and the creation of such images is discussed 
by Mortlock. Mortlock discloses a wire frame which is covered in a texture map (fig 3A, 
column 1 , lines 23-25, column 2, lines 17-52) used to create an image of a human 
head/face. This texture map can be cylindrical (column 20, lines 11-14) among other 
shapes. It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combine the HIP system of Tyree with the 3-D graphics creation technique 
of Mortlock in order to improve the security and therefore the functionality of the system, 
by using non-textual images which would make it difficult for automated scripts which 
are used to attempting to decode the textual image. 

Claim 23 of the application recites: 

A computer-readable medium having computer-executable instructions for 
creating a test to determine whether a user is a person or a bot, said computer 
executable instructions comprising: 

inputting a 3D wire model of a generic head and a texture map of an arbitrary 
person; and 

generating a human interactive proof using said generic head model and texture 
map. 

Claim 23 is essentially the software product for the system of claim 20. It 
discloses no new limitations and as such is rejected for the same reasons as claim 20. 
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Claim 24 of the application recites: 

The computer-readable medium of Claim 23 wherein the human 
interactive proof employs an image of a human face in which certain face 
features are at known locations in said image. 

The limitation of known positions of facial features is disclosed by Mortlock as 
"known points" (column 14, table 1, column 13 lines 20-30). 

Claim 25 of the application recites: 

The computer-readable medium of Claim 24 wherein a comparison of the 
locations of said features input by a user is made to their actual location in the 
image and is used to determine whether the user is a human or a bot. 

Tyree discloses an intelligence test which offers an image to a potential user and 
prompts the user to input the answer to a question about said image ([0102]), which is 
then compared with the stored answer ([0103]). Mortlock discloses known points in the 
facial region such as eyes, ears, nose (column 14, table 1). The location of these points 
in the basic image then would be understandable to the program prior to its distortion. 
After transformations (which are known to the program) are applied, the resultant image 
would not be identifiable by body/facial recognition software of the time. It would have 
been obvious to one of ordinary skill in the art at the time of invention to combine the 
two in order to increase the level of security offered by the "test", thus improving the 
marketability of the software product as a whole. 
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5. Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over Tyree in 
view of Mortlock and further in view of Mori. 

Claim 14 of the application recites: 

The system of Claim 1 0 wherein the features of the face comprise the four 
corners of the eyes and the two corners of the mouth. 

Mortlock discusses a method for creating an image of a human head/face from a 
wire model and a texture map (Fig 3A, column 1, lines 18-21) as well as the known 
points of eyes, ears, chin, etc (column 14, table 1). Regarding the limitation of the facial 
feature of the four corners of the eyes, Mortlock shows texture maps of the two eyes, 
trimmed at the inner and outer corners. In order for such trimming to occur from a larger 
photograph input, it is inherent that the system of Mortlock have the ability to discern the 
position of the four corners of the subject's eyes (Fig 3A #27). Though the chin and 
larynx are disclosed as known locations, the two corners of the mouth are not. The Mori, 
paper discloses an algorithm for locating key points on a human body (abstract, page 5 
section 5, item 1 ). It would have been obvious to one of ordinary skill in the art at the 
time of invention to combine the HIP system of Tyree with the facial image of Mortlock, 
and the feature recognition of Mori, in order to create an HIP that offers a wide range of 
potential answer choices, improving its security and thus its functionality. 
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Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ALLISON W. GAUL whose telephone number is 
(571)270-3616. The examiner can normally be reached on Monday through Friday 7:30 
am to 5:00 pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Charles Kyle can be reached on 571-272-6746. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Allison W Gaul/ 
Examiner, Art Unit 4194 
02/14/2008 

/Charles R. Kyle/ 

Supervisory Patent Examiner, Art Unit 4194 



